Skip to content

Conversation

lliangyu-lin
Copy link
Contributor

Which issue does this PR close?

What changes are included in this PR?

Are these changes tested?

Comment on lines +44 to +45
spark-connect-rs = { git = "https://github.com/apache/spark-connect-rust.git", rev = "061cb3ecb187b039141f20c722c7984e915f3b9d" }
#spark-connect-rs = "0.0.2"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to have some issues with spark-connect-rs 0.0.2 because it's using arrow 53, and that isn't compatible with chrono 0.4.40+, which is a known problem apache/arrow-rs#7196
New version of spark-connect-rs doesn't seem will be release soon, using git url for right now until new version is released.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this block our release? We don't need to publish this crate, just use it in integration tests.

liurenjie1024 added a commit that referenced this pull request Sep 2, 2025
…1630)

## Which issue does this PR close?

- Closes #1214 .

## What changes are included in this PR?
* Add schedule definition
* Parse schedule to engines involved and test steps
* Part of #1621 

## Are these changes tested?

Yes, through unit tests. Some parts, like parse engines, require
integrations and will be tested once engine and catalog support is added
as part of the sql logic tests.


---------

Co-authored-by: liurenjie1024 <[email protected]>
Co-authored-by: Leon Lin <[email protected]>
Ok(Self { engines, steps })
}

async fn parse_engines(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parsing logic maintained manually. Would be nice if we could use ser/de framework for them.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also applied to creation of engines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants